Grabity: A Virtual Reality Haptic Controller for Creating Gravity and Stiffness during Grasping Motions Through Asymmetric vibrations

ABSTRACT

A device to simulate kinesthetic pad opposition grip forces and weight for grasping virtual objects in a virtual reality is provided. The device includes a base, a sliding part, a braking mechanism and a swinging part with linear resonant actuators (e.g. voice coil actuators). The sliding part is connected with the base through a first prismatic joint which allows for single degree of freedom pinching motions for grasping an object. The swinging part connected to the sliding part and the base through revolute joints. The brake mechanism is used to create a grasping force. The linear resonant actuators provide both touch sensation at initial contact and sensation of weight when lifting the object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional PatentApplication 62/571,745 filed Oct. 12, 2017, which is incorporated hereinby reference.

FIELD OF THE INVENTION

This invention relates to virtual reality haptic devices, systems andmethods.

BACKGROUND OF THE INVENTION

To grasp and manipulate objects in the real world, humans rely on hapticcues such as fingertip contact pressure and kinesthetic feedback offinger positions to determine shape, and proprioceptive and cutaneousfeedback for weight perception, among other modalities. Current virtualreality (VR) systems can render realistic 3D objects visually, but mostlack the ability to provide a realistic haptic experience. To createhaptic interfaces that can provide a realistic grasping experience wemust support these same modalities and render similar forces to a user'shands. The present invention addresses these needs.

SUMMARY OF THE INVENTION

The present invention provides a device to simulate kinesthetic padopposition grip forces and weight for grasping virtual objects in avirtual reality. The device includes a base with a thumb pad. The devicefurther includes a sliding part connected with the base through a firstprismatic joint. The sliding part has a finger pad. The prismatic jointallows for single degree of freedom pinching motions between the thumbpad and the finger pad for grasping an object. The device furtherincludes a brake mechanism on the sliding part as a first actuator. Thebrake mechanism is used to create a grasping force. The device alsoincludes a swinging part connected to the sliding part and the basethrough revolute joints. In one example, the swinging part has two voicecoil actuators as second actuators connected through a second prismaticjoint. A first voice coil actuator is placed closely to the finger padand the other voice coil actuator is placed closely to the thumb pad fortransmission of vibration signals. The voice coil actuators provide bothtouch sensation at initial contact and sensation of weight when liftingthe object.

Voice coil actuators is used in the exemplary embodiments but could bereplaced by a one-dimensional linear actuator such as an oscillatingmass on a spring where the position of the mass is controlled by a motoror a voice coil, linear servos, rotary motors with a mechanical linkageto translate rotation to displacement of skin, rotary servo motors witha mechanical linkage to translate rotation to displacement of skin,linear resonant actuators or the like.

For a specific virtual reality application, the base could haveretroreflective markers which could be captured by an external opticalmotion capture system for tracking the thumb's position and orientation.

In some examples, the first and the second prismatic joints are made ofcarbon fiber tubes, the finger pad is an index finger pad, and the brakemechanism could contain a brake lever, a tendon, and a motor. In someother examples, the offset distance between the revolute joint andcenter of mass of the swinging part could ensure that the direction ofthe voice coil actuators is always passively directed to be normal toground. In yet some other examples, the second prismatic joint on theswinging part could constrain two angles of the voice coil actuators tobe the same while allowing them to slide relative to one another.

Embodiments of the invention have advantages in that (i) they provideweight sensations to a mobile haptic device by creating asymmetricvibrations, and (ii) the bearing mechanism reduces the number of voicecoil actuators for weight simulation; i.e. by attaching linear resonantactuators (e.g. voice coil actuators) through bearings, the vibrationsfrom the linear resonant actuators are always normal to ground, and thisreduces the number of actuators for weight simulation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-B show according to an exemplary embodiment of the invention adevice to simulate kinesthetic pad opposition grip forces and weight forgrasping virtual objects in a virtual reality.

FIG. 2 shows according to an exemplary embodiment of the invention asystem diagram for the device shown in FIGS. 1A-B.

FIG. 3 shows according to an exemplary embodiment of the invention acurrent signal generating asymmetric vibration. A is amplitude ofcurrent.

FIG. 4 shows according to an exemplary embodiment of the invention ablock diagram of a virtual reality application.

DETAILED DESCRIPTION

In this invention, we introduce Grabity, a mobile, ungrounded hapticdevice, which can display various types of kinesthetic feedback toenhance the grasping and manipulating of virtual objects. This feedbackincludes gravity, force for inertia, as well as rigid stiffness forcefeedback between opposing fingers. The design combines a “gripper” stylehaptic device, for providing opposing forces between the index fingerand thumb, and a skin deformation mechanism, for rendering inertia andmass of a virtual object. Asymmetric skin deformation enabled by linearvibratory motors can generate perceived virtual forces tangential tofinger pads.

In this invention, we apply this principle to render the virtual gravityforce of different virtual masses, and their associated inertia in a 1degree of freedom per finger. To create the sensation of gravity andinertia, we adapt two voice coil actuators to a mobile gripper typehaptic device. We utilize different magnitudes of asymmetric vibrationsto generating various levels of force feedback. The gripper elementincludes a unidirectional brake to create the rigid, high-stiffness,opposing forces between a finger and thumb.

To design a device that can provide feedback for touching, grasping,gravity, and inertia, we chose to combine a gripper type device withcutaneous asymmetric skin stretch. For this device's performance for ourscenarios, we emphasize the optimization of the following designparameters:

-   -   Weight. The overall mass of the device should be lightweight in        order for the skin stretch to be perceived as weight, as the        virtual forces have been shown to be low (<30 g) and weight        perception acuity decreases as total weight increases.    -   Motion range. Wide range of motion to grasp and manipulate        different sized objects.    -   Mechanical complexity. Minimize the number of actuators, to        reduce cost and weight. In addition, research has yet to show        that people can simultaneously integrate multiple directions of        asymmetric skin stretch well.    -   Anatomical alignment. The index finger and thumb should be        parallel in alignment to receive consistent directional skins        stretch cues. Misalignment can create confusion and        unintentional torques.    -   Stiffness. High stiffness for pad opposition forces. Grip force        of the human hand can exceed 100N.    -   Performance. Accurate and fast position tracking to integrate        into VR.

Overall Structure

As shown in FIGS. 1A-B, the device distinguishes three rigid bodies: abase, a sliding part, and a swinging part. The base is mounted on thethumb, and it has retroreflective markers for an external optical motioncapture system for tracking the thumb's position and orientation. Forthe purposes of the invention, a global position tracking system isrequired if one wants to use it for a VR application. In one example, weused retroreflective markers as a passive position tracking system. LEDscould be mounted for an active tracking system, or a software algorithmcalculating position and orientation through camera images. However, theposition tracking system is not directly related to the weightsimulating technology. The sliding part is mounted on the index fingerand is connected with the base through a prismatic joint that iscomposed of two carbon fiber tubes. The sliding part could also bemounted on another finger. This single degree of freedom allows pinchingmotions for grasping objects. In one example, a brake mechanism on thesliding part contains a brake lever, a tendon, and a motor. However,such specific braking mechanism is not necessary for the implementationof the invention. Any mechanism would be possible if it provides forcefeedback between the thumb and fingers so that the fingertip skins havesolid contact (normal force) with the haptic device. The swinging part,which is connected to the sliding part and base through revolute joints,is has two voice coil actuators (e.g. Haptuator Mark II, Tactile Labs)and a prismatic joint made of carbon fiber tubes. Each voice coilactuator is placed closely to the index finger and thumb pads so that itcan transmit the desired vibration signals properly. The offset distancebetween the revolute joint and center of mass of the swinging partensures that the direction of the voice coil actuators is alwayspassively directed to be normal to ground. The prismatic joint on theswinging part constrains the two angles of voice coil actuators to bethe same while allowing them to slide relative to one another. In thisexemplary embodiment, most parts of the device are 3D printed using aFormlabs 2 printer (SLA technology), and the device weighs 65 grams.

Actuation for Force Feedback

The Grabity device contains two types of actuators: a brake mechanismand two voice coil actuators. The brake mechanism is used to create arigid grasping force, while the voice coil actuators provide both touchsensation at initial contact and the sensation of weight when liftingthe object. FIG. 2 shows a system diagram of Grabity.

Touching: Conventional Vibration

When a user touches a virtual object without grasping, the voice coilactuators act like conventional vibrotactile transducers and play simplesymmetric vibrations to indicate the point of initial contact. The voicecoil actuator on the index finger or the thumb vibrates individuallydepending on which finger is touching the virtual object.

Grasping: Unidirectional Brake Mechanism

When a user grasps a virtual object, the brake mechanism is activated tocreate a rigid passive force, which is an adapted unidirectional brakemechanism. In an exemplary embodiment, the brake mechanism provides alocking force using a brake lever, which is activated by a small DCmotor (6 mm). Once the brake is engaged, the motor is turned off and theuser's own grasping force keeps the brake lever engaged. While the brakeis engaged it provides strong resistance that exceeds 100 N in thedirection of the two fingers. However, when the user releases theirgrip, the brake disengages, allowing the user to open her hand. Onecould use a rubber tendon to move the brake lever back to the originalposition when releasing a grip, but here we use two magnets (one on thebrake lever and the other on the body of the device) to reset the lever,for more consistent and reliable performance.

Weight: Asymmetric Vibration

When a user lifts or shakes a virtual object, the voice coil actuatorsvibrate asymmetrically to generate the sensation of weight. If themagnet inside the voice coil actuator moves down quickly and moves upslowly, the skin on the user's finger pads is stretched asymmetrically.

FIG. 3 shows the shape of commanded current pulses. The shape of thesecurrent pulses is designed to give the actuators a large step of currentinitially to cause a large acceleration in the magnet, then to ramp thecurrent back down to slowly return that magnet to its starting position.To achieve this asymmetric actuation, a fast analog signal and currentcontroller are required. Therefore, in this exemplary embodiment we useda Teensy 3.6 microcontroller (ARM Cortex-M4 at 180 Mhz with two DACs)generating 15 kHz analog signal output and a linear current-drivecircuit. A current-drive circuit creates less effective damping to thesystem than a voltage-drive circuit, so it is more suitable forasymmetric vibration control. We chose a drive signal with a 40 Hzfrequency (25 ms period) and 0.3 pulse width ratio (t₁=7.5 ms andt₂=17.5 ms). To simulate various weight sensations, we change theamplitude A of the signal, while keeping the frequency and pulse widthratio fixed.

Sensing

An OptiTrack motion tracking system is used to track the position andorientation of the thumb. A magnetic encoder is attached to the indexfinger sliding assembly to track it's position relative to the thumb.The encoder is friction driven and rolls on the prismatic joint (carbonfiber tube). Using the data from the motion tracking system and magneticencoder, we can render user's thumb and index finger in VR.

The resolution of the magnetic encoder and friction drive assembly wasmeasured to be 2 mm of linear travel. This resolution is sufficientbased on just noticeable difference (JND) results of fingers to thumbdistance perception. The Wolverine system had a Time-of-Flight sensor tomeasure this distance with a resolution of 1 mm; however, it also had ±1mm noise with 100 Hz sampling rate. By adapting the magnetic encoder, wecan achieve much lower noise and much higher sampling rate (kHz), at thecost of resolution.

Software: Virtual Reality Haptics Framework

The software framework for this exemplary embodiment was implemented inC++ and uses multiple software libraries. As a virtual haptic device,Grabity requires knowledge of its position as input and produces forceas output. The information flow begins with position tracking of Grabitywith the OptiTrack motion tracking system. The framework gets the 6DOFpose through the Motive C++ API. The grasping distance is transferredover the Teensy's serial link (250,000 bits per second) from the encoderin the Grabity device's slider.

In CHAI3D (version 3.2.0), Grabity is represented as a subclass of thecGenericHapticDevice that accesses both the device position and thegrasping distance. CHAI3D integration for Open Dynamics Engine (ODE)renders the physical interactions. The display appears on the Oculus VRheadset, and the force output given by ODE is passed along to thecGrabityDevice class. The force is further separated into its componentsas it is to the Teensy microcontroller.

Mass Simulation During Grasping

CHAI3D provides the output force, torque, and gripper force to thecustom haptic device. Because Grabity has only two modes of actuation,these virtual values need to be converted into a voice coil signal and acommand to lock the slider. The locking occurs when the gripper force(the force pushing apart the thumb and finger, i.e., from gripping ablock) is greater than an empirically determined threshold of 0.7 N.This value was chosen to avoid locking when one finger strikes a block,but trigger locking quickly when a block is grasped.

Determining the voice coil signal is more complex. First, we must assumethe voice coil is always pointing downward. This assumption is notalways correct, as the coils swing on a limited range and only in onedimension. However, we have found that most hand orientations thesubjects use are sufficiently close to this approximation. As such, weuse the output force's z-component and ignore the other two.

Second, we must separately extract the downward force for each of thetwo fingers. This information is encoded in the torque. As previously,we approximate the voice coil directions as downward. We thus projectthe finger-to-thumb vector to the ground plane, and use that vector toconvert torque back to force, and add it to the z-output-force. Thisforce value is transmitted over the serial link to the Teensymicrocontroller.

In the Teensy, the force is mapped to a voice coil signal. We use thedata from the first user study, below, to construct the mapping fromamplitude to virtual force. The amplitude of the signal is capped sothat forces larger than can be expressed by the voice coil are expressedby the maximum perceived force.

For evaluation studies, details and performance results the reader isdirected to U.S. Provisional Patent Application 62/571,745 filed Oct.12, 2017, which is incorporated herein by reference.

What is claimed is:
 1. A device to simulate kinesthetic pad oppositiongrip forces and weight for grasping virtual objects in a virtualreality, comprising: (a) a base, wherein the base has a thumb pad; (b) asliding part connected with the base through a first prismatic joint,wherein the sliding part has a finger pad, wherein the prismatic jointallows for single degree of freedom pinching motions between the thumbpad and the finger pad for grasping an object, (c) a brake mechanism onthe sliding part as a first actuator, wherein the brake mechanism isused to create a grasping force; and (c) a swinging part connected tothe sliding part and the base through revolute joints, wherein theswinging part comprises two linear resonant actuators as secondactuators connected through a second prismatic joint, wherein one linearresonant actuator is placed closely to the finger pad and the otherlinear resonant actuator is placed closely to the thumb pad fortransmission of vibration signals, and wherein the linear resonantactuators provide both touch sensation at initial contact and sensationof weight when lifting the object.
 2. The device as set forth in claim1, wherein the base has retroreflective markers to be captured by anexternal optical motion capture system for tracking the thumb's positionand orientation.
 3. The device as set forth in claim 1, wherein thefirst and the second prismatic joints are made of carbon fiber tubes. 4.The device as set forth in claim 1, wherein the finger pad is an indexfinger pad.
 5. The device as set forth in claim 1, wherein the brakemechanism contains a brake lever, a tendon, and a motor.
 6. The deviceas set forth in claim 1, wherein an offset distance between the revolutejoint and center of mass of the swinging part ensures that the directionof the linear resonant actuators is always passively directed to benormal to ground.
 7. The device as set forth in claim 1, wherein linearresonant actuators are voice coil actuators.
 8. The device as set forthin claim 1, wherein the second prismatic joint on the swinging partconstrains two angles of linear resonant actuators to be the same whileallowing them to slide relative to one another.